Lecture 09: Charts¶
Associated Textbook Sections: 7.0, 7.1
Overview¶
Set Up the Notebook¶
from datascience import *
import numpy as np
%matplotlib inline
import matplotlib.pyplot as plt
plt.style.use('fivethirtyeight')
W. E. B. Du Bois¶
Background¶
The content of the following quotes, podcast/video, and images contains references to the enslavement and murder of humans.
- Scholar, historian, activist, and data scientist
"The Philadelphia Negro was the first scientific study of race in the world. [...] the first non-racist investigation of a non-white poulation in the world. [...] one of the first social scientific written in the U.S. using the advanced statistical methods of the time." - Dr. Tukufu Zuberi, Professor of Race Relations at the University of Pennsylvania (Source: A Legacy of Courage: W.E.B. Du Bois and the Philadelphia Negro)
- First Black American to receive a PhD from Harvard
- NAACP founder
Paris Exposition¶
Made a series of visualizations for the 1900 Paris Exposition
- Goal: Change the way people see Black Americans
- Hundreds of photographs and patents
- 60+ handmade graphs in 3 months
"All art is propaganda, and ever must be, despite the wailing of the purists. I stand in utter shamelessness and say that whatever art I have for writing has been used always for propaganda for gaining the right of black folk to love and enjoy. I do not care a damn for any art that is not used for propaganda." - W.E.B. Du Bois
- Ideologies of W.E.B. Du Bois and Booker T. Washington are typically compared
- The following podcast provides an 11-minute overview of Du Bois and Washington:
from IPython.display import IFrame
IFrame('https://www.youtube-nocookie.com/embed/zHn-vSTMOWE?si=Qk49xkLYJyAlBY5k',
width="560", height="315", frameBorder="0", allowfullscreen="",
allow="accelerometer; autoplay; clipboard-write; \
encrypted-media; fullscreen; gyroscope; picture-in-picture; web-share",
loading="lazy", title="YouTube video player",
referrerpolicy="strict-origin-when-cross-origin")
Images from Paris Exposition¶
Why Do We Visualize Data¶
- A large fraction of our brains are dedicated to visual reasoning.
- In Data Science we use visualization:
- For others – to communicate our findings
- For ourselves – to understand our data, see patterns, and discover relationships
Course Visualizations¶
- In the course we will mostly use the following visualizations:
- Histograms
- Line Graphs
- Scatter Plots
- Bar Charts
- You will need to overlay graphs to explore relationships
- How you visualize your data depends on attribute type
- The data type doesn't determine numerical/categorical attribute label.
'$12.00'is astrand likely to reflect a numerical attribute- The context of the data and analysis is important to understand
You will indirectly work with the standard Matplotlib library for data visualization using the datascience library. You can optionally interact with visualizations using the Plotly library, but customizing and creating interactive visualizations is not required and you will not be tested on these things.
Good Practices¶
- Less can be more
- Minimize decoration
- Choose colors carefully: Minimize the number of different colors
- If data are numerical, preserve their relative values and distances between them
See Edward Tufte's "The Visual Display of Quantitative Information" for additional suggestions.
Demo: Identifying Data Type of Column Values¶
The dataset top_grossing_movies.csv shows the highest 100-grossing movies worldwide listed on IMDB. Adjusted total gross values were also provided using the Consumer Price Index (CPI)-based Python library cpi to account for inflation.
top_movies = Table.read_table('top_grossing_movies.csv')
top_movies
| Created | Modified | Title | URL | Title Type | IMDb Rating | Runtime (mins) | Year | Genres | Num Votes | Release Date | Directors | Gross | Gross (Adjusted) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2/3/23 | 2/9/23 | Avatar | https://www.imdb.com/title/tt0499549/ | Movie | 7.9 | 162 | 2009 | Action, Adventure, Fantasy, Sci-Fi | 1402235 | 12/18/09 | James Cameron | 2923706026 | 4.15247e+09 |
| 2/3/23 | 8/9/23 | Avengers: Endgame | https://www.imdb.com/title/tt4154796/ | Movie | 8.4 | 181 | 2019 | Action, Adventure, Drama, Sci-Fi | 1298188 | 4/26/19 | Anthony Russo, Joe Russo | 2799439100 | 3.33648e+09 |
| 2/3/23 | 5/23/23 | Avatar: The Way of Water | https://www.imdb.com/title/tt1630029/ | Movie | 7.5 | 192 | 2022 | Action, Adventure, Fantasy, Sci-Fi | 509990 | 12/16/22 | James Cameron | 2320246732 | 2.41576e+09 |
| 2/3/23 | 3/13/23 | Titanic | https://www.imdb.com/title/tt0120338/ | Movie | 7.9 | 194 | 1997 | Drama, Romance | 1304263 | 12/19/97 | James Cameron | 2255734210 | 4.28241e+09 |
| 2/3/23 | 2/3/23 | Star Wars: Episode VII - The Force Awakens | https://www.imdb.com/title/tt2488496/ | Movie | 7.8 | 138 | 2015 | Action, Adventure, Sci-Fi | 983853 | 12/18/15 | J.J. Abrams | 2071310218 | 2.66281e+09 |
| 2/3/23 | 2/3/23 | Avengers: Infinity War | https://www.imdb.com/title/tt4154756/ | Movie | 8.4 | 149 | 2018 | Action, Adventure, Sci-Fi | 1232719 | 4/27/18 | Anthony Russo, Joe Russo | 2052415039 | 2.49047e+09 |
| 2/3/23 | 6/7/23 | Spider-Man: No Way Home | https://www.imdb.com/title/tt10872600/ | Movie | 8.2 | 148 | 2021 | Action, Adventure, Fantasy, Sci-Fi | 905126 | 12/17/21 | Jon Watts | 1921847111 | 2.16109e+09 |
| 2/3/23 | 6/7/23 | Jurassic World | https://www.imdb.com/title/tt0369610/ | Movie | 6.9 | 124 | 2015 | Action, Adventure, Sci-Fi | 688018 | 6/12/15 | Colin Trevorrow | 1671537444 | 2.14888e+09 |
| 2/3/23 | 2/3/23 | The Lion King | https://www.imdb.com/title/tt6105098/ | Movie | 6.8 | 118 | 2019 | Animation, Adventure, Drama, Family, Fantasy, Musical | 271300 | 7/19/19 | Jon Favreau | 1663075401 | 1.98212e+09 |
| 2/3/23 | 2/3/23 | The Avengers | https://www.imdb.com/title/tt0848228/ | Movie | 8 | 143 | 2012 | Action, Sci-Fi | 1477106 | 5/4/12 | Joss Whedon | 1518815515 | 2.01567e+09 |
... (90 rows omitted)
The movie titles reflect a categorical attribute.
type(top_movies.column('Title').item(0))
str
The movie years reflect a numerical attribute.
type(top_movies.column('Year').item(0))
int
Be careful. Sometimes the data type doesn't align with the intended attribute type.
Categorical Data¶
(Horizontal) Bar charts barh are a standard way to visualize the distribution of a single categorical variable.
A Bar Chart¶
The following code uses group. We will address that later in the course. Additionally, there is customization to the visual done on the lines that start with plots. You are not responsible for this customization.
cones = Table().read_table('cones.csv')
cones_grouped_by_flavor = cones.group('Flavor')
cones_grouped_by_flavor.barh('Flavor')
plt.title('Distrubtion of Ice Cream Flavors')
plt.show()
Demo: Bar Charts¶
Reduce the table to the top 10 movies based on actual gross values ('Gross (Adjusted)') for the movies released in the last decade.
top_movies_select = top_movies.select('Title', 'Year', 'Gross (Adjusted)')
top_movies_last_decade = top_movies_select.where('Year', are.above(2014))
top_movies_last_decade_sorted = top_movies_last_decade.sort('Gross (Adjusted)', True)
top10 = top_movies_last_decade_sorted.take(np.arange(10)) # SOLUTION
top10
| Title | Year | Gross (Adjusted) |
|---|---|---|
| Avengers: Endgame | 2019 | 3.33648e+09 |
| Star Wars: Episode VII - The Force Awakens | 2015 | 2.66281e+09 |
| Avengers: Infinity War | 2018 | 2.49047e+09 |
| Avatar: The Way of Water | 2022 | 2.41576e+09 |
| Spider-Man: No Way Home | 2021 | 2.16109e+09 |
| Jurassic World | 2015 | 2.14888e+09 |
| The Lion King | 2019 | 1.98212e+09 |
| Furious 7 | 2015 | 1.94808e+09 |
| Avengers: Age of Ultron | 2015 | 1.80625e+09 |
| Frozen II | 2019 | 1.73256e+09 |
Returning to top grossing movies data, convert to the gross (adjusted) values in the top10 table to billions of dollars for readability.
billions = np.round(top10.column('Gross (Adjusted)') / 1000000000, 2)
top10 = top10.with_column('Gross Adjusted (Billions)', billions)
top10
| Title | Year | Gross (Adjusted) | Gross Adjusted (Billions) |
|---|---|---|---|
| Avengers: Endgame | 2019 | 3.33648e+09 | 3.34 |
| Star Wars: Episode VII - The Force Awakens | 2015 | 2.66281e+09 | 2.66 |
| Avengers: Infinity War | 2018 | 2.49047e+09 | 2.49 |
| Avatar: The Way of Water | 2022 | 2.41576e+09 | 2.42 |
| Spider-Man: No Way Home | 2021 | 2.16109e+09 | 2.16 |
| Jurassic World | 2015 | 2.14888e+09 | 2.15 |
| The Lion King | 2019 | 1.98212e+09 | 1.98 |
| Furious 7 | 2015 | 1.94808e+09 | 1.95 |
| Avengers: Age of Ultron | 2015 | 1.80625e+09 | 1.81 |
| Frozen II | 2019 | 1.73256e+09 | 1.73 |
Visualize the gross adjusted values for each of the top 10 grossing (adjusted) movies.
top10.barh('Title', 'Gross (Adjusted)')
plt.title("The Top 10 Grossing Movies")
plt.show()
Visual Perception Accuracy¶
From Nathan Yau’s Data Points: Visualization that Means Something, our eyes can extract information at different levels of accuracy depending on the design.
For this reason, pie charts are generally discouraged because most people have a difficult time visually interpreting angles compared to lengths of bars.
Demo: Visualizing Du Bois¶
Read the du_bois.csv data as a table, reformat the data, and create a stacked bar chart.
du_bois = Table.read_table('du_bois.csv')
du_bois.set_format('RENT', PercentFormatter)
du_bois.set_format('FOOD', PercentFormatter)
du_bois.set_format('CLOTHES', PercentFormatter)
du_bois.set_format('TAXES', PercentFormatter)
du_bois.set_format('OTHER', PercentFormatter)
du_bois
| CLASS | ACTUAL AVERAGE | RENT | FOOD | CLOTHES | TAXES | OTHER | STATUS |
|---|---|---|---|---|---|---|---|
| 100-200 | 139.1 | 19.00% | 43.00% | 28.00% | 0.10% | 9.90% | POOR |
| 200-300 | 249.45 | 22.00% | 47.00% | 23.00% | 4.00% | 4.00% | POOR |
| 300-400 | 335.66 | 23.00% | 43.00% | 18.00% | 4.50% | 11.50% | FAIR |
| 400-500 | 433.82 | 18.00% | 37.00% | 15.00% | 5.50% | 24.50% | FAIR |
| 500-750 | 547 | 13.00% | 31.00% | 17.00% | 5.00% | 34.00% | COMFORTABLE |
| 750-1000 | 880 | 0.00% | 37.00% | 19.00% | 8.00% | 36.00% | COMFORTABLE |
| 1000 and over | 1125 | 0.00% | 29.00% | 16.00% | 4.50% | 50.50% | WELL-TO-DO |
Notice that the table is formatted to show percentages, but the values in the % columns are actually floats.
du_bois.column('RENT')
type(du_bois.column('RENT').item(0))
For a quick review, find the income bracket (CLASS) that spent the highest percentage of their income on rent.
...
Start to re-create the bar chart that Du Bois presented in Paris.
du_bois_for_bar = du_bois.drop('ACTUAL AVERAGE', 'STATUS')
du_bois_for_bar
| CLASS | RENT | FOOD | CLOTHES | TAXES | OTHER |
|---|---|---|---|---|---|
| 100-200 | 19.00% | 43.00% | 28.00% | 0.10% | 9.90% |
| 200-300 | 22.00% | 47.00% | 23.00% | 4.00% | 4.00% |
| 300-400 | 23.00% | 43.00% | 18.00% | 4.50% | 11.50% |
| 400-500 | 18.00% | 37.00% | 15.00% | 5.50% | 24.50% |
| 500-750 | 13.00% | 31.00% | 17.00% | 5.00% | 34.00% |
| 750-1000 | 0.00% | 37.00% | 19.00% | 8.00% | 36.00% |
| 1000 and over | 0.00% | 29.00% | 16.00% | 4.50% | 50.50% |
du_bois_for_bar.barh('CLASS')
# Some extra graph formatting you are not responsible for
plt.title('W.E. Du Bois Income and Expenditure')
plt.show()
[Optional] Interactive Charts with Plotly¶
- By default, we will be using the static visualizations that are made using the Matplotlib library.
- You have the ability to access interactive Plotly visualizations by adding an
iin front of the table method name that creates the default visual. - The arguments change to fit the Plotly functions.
[Optional] Demo: Visualizing Du Bois with Plotly¶
Create the interactive version of the bar chart.
du_bois_for_bar.ibarh(
column_for_categories='CLASS',
title='W.E. Du Bois Income and Expenditure',
xaxis=dict(tickformat='0.1%')
)
Plotly has an easy way to stack the bars to create an overlaid bar chart.
# barmode and xaxis are available with ibarh because they are a Plotly arguments
fig = du_bois_for_bar.ibarh(
column_for_categories='CLASS',
barmode="stack",
title='W.E. Du Bois Income and Expenditure',
xaxis=dict(tickformat='0.1%')
)
We are starting to get something that looks like Du Bois's visual, but let's stop there because this is optional for this class. If you like creating visualizations, try to read through the Plotly documentation or Matplotlib documentation to update the colors, add overlaid text, etc.
Numerical Data¶
Visualizing the Distribution of One Numerical Variable¶
Histograms tbl.hist are a standard way to visualize the distribution of one numerical variable.
Histograms will be focused on in the next lecture.
A Histogram¶
top_movies.hist('Year', unit="Year")
# Some extra graph formatting you are not responsible for
plt.title('Distribution of Total Gross')
plt.show()
Plotting Two Numerical Variables¶
Line graphs tbl.plot and Scatter plots tbl.scatter are standard ways to visualize the relationship of two numerical variables.
A Line Graph¶
movies_per_year = top_movies.group('Year').relabeled('count', 'Number of Movies')
movies_per_year.where('Year', are.above(1999)).plot('Year', 'Number of Movies')
# Some extra graph formatting you are not responsible for
plt.xticks(np.arange(2000, 2023, 5))
plt.title('Number of Movies vs. Release Year')
plt.show()
A Scatter Plot¶
actors = Table.read_table('actors.csv')
actors.scatter('Number of Movies', 'Average per Movie')
# Some extra graph formatting you are not responsible for
plt.title('Average Pay per Movie (Thousands of Dollars) vs. Number of Movies')
plt.show()
When to use a line vs scatter plot?¶
- Use line plots for sequential data if:
- ... your x-axis has an order
- ... sequential differences in y values are meaningful
- ... there's only one y-value for each x-value
- Usually: x-axis is time or distance
- Use scatter plots for non-sequential data --- When you’re looking for associations
Demo: Scatter Plots¶
Visualize the relationship between the IMDb rating and the number of votes for the movies in top_movies.
top_movies.show(3)
| Created | Modified | Title | URL | Title Type | IMDb Rating | Runtime (mins) | Year | Genres | Num Votes | Release Date | Directors | Gross | Gross (Adjusted) |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2/3/23 | 2/9/23 | Avatar | https://www.imdb.com/title/tt0499549/ | Movie | 7.9 | 162 | 2009 | Action, Adventure, Fantasy, Sci-Fi | 1402235 | 12/18/09 | James Cameron | 2923706026 | 4.15247e+09 |
| 2/3/23 | 8/9/23 | Avengers: Endgame | https://www.imdb.com/title/tt4154796/ | Movie | 8.4 | 181 | 2019 | Action, Adventure, Drama, Sci-Fi | 1298188 | 4/26/19 | Anthony Russo, Joe Russo | 2799439100 | 3.33648e+09 |
| 2/3/23 | 5/23/23 | Avatar: The Way of Water | https://www.imdb.com/title/tt1630029/ | Movie | 7.5 | 192 | 2022 | Action, Adventure, Fantasy, Sci-Fi | 509990 | 12/16/22 | James Cameron | 2320246732 | 2.41576e+09 |
... (97 rows omitted)
top_movies.scatter('IMDb Rating', 'Num Votes')
# Some extra graph formatting you are not responsible for
plt.title('IMDb Rating vs. Number of Votes')
plt.show()
[Optional] Demo: Interactive Scatter Plots¶
Again, for all the visualization methods we use from the datascience library, if you put an i infront of the name of the visualization, you can access an interactive version of plot that is based on another visualization library called Plotly. You will not be tested on your knowledge of these interactive plots. You might find them helpful for exploring the data.
top_movies.iscatter(
column_for_x='IMDb Rating',
select='Num Votes',
labels='Title',
title='IMDb Rating vs. Number of Votes'
)